CCD's and CMOS's are used as imaging elements in digital cameras. In order to respond to a demand for higher image quality, efforts have been made to reduce pixel size and increase the number of pixels. While the reduction of the pixel size can be achieved by reducing the size of the photodiode and vertical direction (V) transfer path, reduction of the size of the photodiode and the V transfer path increases the probability of generation of defects. A localized crystal defect of a photodiode causes pixel degradation and defects in which a constant bias voltage is always applied to an opto-electric conversion output corresponding to an amount of incident light causes a white spot with a high brightness on the monitor, and thus causes a defect which is commonly called a white flaw. A defect in which the photosensitivity of the photodiode is reduced appears on the monitor as a black point, and thus causes a defect which is commonly called a black flaw. A defect in which a dust or like is attached to the V transfer path blocks transfer of charges and appears on the monitor as a linear flaw, and thus causes a defect which is commonly called a vertical flaw. Therefore, it is necessary to correct the point flaw and the vertical flaw. The point flaw and the vertical flaw can be corrected through a correction using pixel values of pixels surrounding the defect and in which the point defect and V transfer path defect are not present.
U.S. Pat. No. 6,741,754 discloses a technique for correcting a point flaw and a vertical flaw. This related art will now be described. FIG. 49 shows a flowchart of an overall process in this related art. A target image is imaged on a CCD through an optical system of the digital camera such as a lens, a shutter, and a diaphragm. Each photodiode which forms a part of the CCD converts light from the target into an electrical signal based on the amount of light and outputs as an image signal. The image signal is converted into a digital signal by an A/D device and then supplied to an image processor. The image processor comprises a system LSI and processes a digital image signal to generate image signals of R, G, and B. The image signals of R, G, and B are converted into an analog signal by a A/D device and displayed on a display panel on a back surface of the digital camera. When the user presses a shutter button of the digital camera, the image captured at the time of pressing is output from the image processor and stored in a memory. The image processor determines whether or not there is a point flaw or a vertical flaw within the captured image at the timing of the pressing operation of the button. When at least one point flaw or vertical flaw is present, any point flaw is first corrected (S101) and then, after the point flaw is corrected, any vertical flaw is corrected. In the correction of the vertical flaw, the vertical flaw of the G pixel is first corrected (S102) and then the vertical flaw of the R pixel and the vertical flaw of the B pixel are corrected (S103). After the point flaw and the vertical flaw are corrected, normal signal processes are applied and the signal is stored in a memory (S104).
FIG. 50 shows a method of correcting the point flaw in the G pixel. In FIG. 50, it is assumed that the point defect appears in a pixel G23. Four pixels surrounding the G23 pixel, that is, a pixel G12, a pixel G14, a pixel G32, and a pixel G34 are used to interpolate a pixel value of the pixel G23 to correct the defect using:G23=(G12+G14+G32+G34)/4
The above-described expression indicates that the pixel G23 is corrected using an average of surrounding four pixels.
FIG. 51 shows a method for correcting a point flaw of an R pixel. In FIG. 51, it is assumed that a point defect appears in a pixel R22. Four pixels surrounding the pixel R22, that is, a pixel R02, a pixel R42, a pixel R20, and a pixel R24 are used to correct the pixel value of the R22 pixel using:R22=(R02+R42+R20+R24)/4Because the placement of the B pixel is similar to that of the R pixel, the point defect of the B pixel can be corrected in a manner similar to the point effect of the R pixel.
FIG. 52 shows a vertical flaw of the G pixel. FIG. 52 shows a case in which a defect appears in the transfer path along the V direction including the pixel G23 and a vertical line including G23 becomes black.
FIG. 53 is a block diagram of a vertical flaw correction circuit for G which corrects the vertical flaw shown in FIG. 52. A correction circuit 10 is provided within an image processor of the digital camera and comprises an interpolation pattern unit 12, a vertical line flaw detection pattern unit 14, and a calculation pattern selector 16. The interpolation pattern unit 12 executes an interpolation calculation for all of a plurality of correction patterns which are defined in advance. The vertical line flaw detection pattern unit 14 detects the pattern of the vertical line flaw through calculation. The calculation pattern selector 16 selects one of the correction patterns based on the calculation result form the vertical line flaw detection pattern unit 14 and outputs the selected correction value as a correction value of the G pixel.
FIG. 54 shows a calculation in the vertical line flaw detection pattern unit 14. The vertical line flaw detection pattern unit 14 executes four calculations shown in FIG. 54. FIG. 54(a) shows a pattern in which a difference between an upper-right pixel G14 and a lower-left pixel G32 which are adjacent to the pixel G23 to be corrected is calculated. When the difference calculation in this direction is abbreviated as Sla:G(Sla)=ABS(G14−G32)
In this expression, “ABS” indicates an absolute value. FIG. 54(b) shows a pattern in which a difference between an upper-left pixel G12 and a lower-right pixel G32 which are adjacent to the pixel G23 to be corrected is calculated. When the difference calculation in which direction is abbreviated as Bac:G(Bac)=ABS(G12−G34)
FIG. 54(c) shows a pattern in which a difference between pixels G21 and G25 which are pixels immediately to the right and to the left in the horizontal direction of the pixel G23 to be corrected is calculated. When the difference calculation in this direction is abbreviated as Hor:G(Hor)=ABS(G21−G25)
FIG. 54(d) shows a pattern in which a difference among pixels G12, G32, G14, and G34 which are adjacent pixels of the pixel G23 to be corrected along the vertical direction is calculated. When the difference calculation in this direction is abbreviated as G(Ver):G(Ver)=ABS(G12+G14−G32-G34)
The vertical line flaw detection pattern unit 14 executes these four difference calculations and supplies the calculated values to the calculation pattern selector 16.
FIG. 55 shows a calculation performed at the interpolation pattern unit 12. FIG. 55(a) shows a correction calculation using the upper-right pixel, lower-right pixel, upper-left pixel, and lower-left pixels which are adjacent to the pixel G23 to be corrected. This calculation can be represented as:G23=(G12+G14+G32+G34)/4
FIG. 55(b) shows a correction calculation using pixels G21 and G25 which are adjacent to the pixel G23 to be corrected along the horizontal direction. This calculation can be represented as:G23=(G21+G25)/2
FIG. 55(c) shows a correction calculation using the upper-right pixel G14 and the lower-left pixel G32 which are adjacent to the pixel G23 to be corrected. This calculation can be represented as:G23=(G14+G32)/2
FIG. 55(d) shows a correction calculation using the upper-left pixel G12 and the lower-right pixel G34 which are adjacent to the pixel G23 to be corrected. This calculation can be represented as:G23=(G12+G34)/2
The interpolation pattern unit 12 supplies correction values which are corrected using these four correction patterns to the calculation pattern selector 16. The calculation pattern selector 16 selects one of the four correction values from the interpolation pattern unit 12 based on a magnitude comparison among four difference calculation values from the vertical line flaw detection pattern unit 14 and outputs the selected correction value. More specifically, the calculation pattern selector 16 calculates a calculation pattern having a small difference calculation value and selects a correction pattern corresponding to the calculation pattern. Based on the four difference calculation values of G(Sla), G(Bac), G(Hor), and G(Ver), the calculation pattern selector 16 determines:G(Hor)<G(Sla) and G(Hor)<G(Bac)   (1)
When the difference in the pixels along horizontal direction is small and this condition is satisfied, the calculation pattern selector 16 determines:G(Ver)<threshold value   (2)
When the difference in the pixels along vertical direction is small and this condition is also satisfied, it is determined that the difference value is small both in the horizontal direction and in the vertical direction. Therefore, a correction pattern shown in FIG. 55(a) is selected. When, on the other hand, the condition (1) is satisfied, but condition (2) is not satisfied, it is determined that the difference value in the vertical direction is large and the correction pattern of FIG. 55(b) is selected in order to execute the correction process only with the pixels of the horizontal direction.
When the condition (1) is not satisfied, the calculation pattern selector 16 determines:G(Sla)<G(Bac)   (3)
When the difference value in the Sla direction is small, the correction pattern of FIG. 55(c) is selected. On the contrary, when the condition (3) is not satisfied, the correction pattern of FIG. 55(d) is selected. In summary, this process is an algorithm in which the target pixel is corrected using surrounding pixels in which the difference calculation value is small and thus the pixel values do not significantly differ from each other.
FIG. 56 is a block diagram of a vertical line flaw correction circuit for R. A correction circuit 18 comprises an interpolation pattern unit 20, a vertical line flaw detection pattern unit 22, and a calculation pattern selector 24. The functions of these units are similar to the interpolation pattern unit 12, vertical line flaw detection pattern section 14, and calculation pattern selector 16 of the correction circuit 10.
FIG. 57 shows a calculation in the vertical line flaw detection pattern unit 22. FIG. 57(a) is a difference calculation using pixels G12, G21, G23, and G32 surrounding a pixel R22 to be corrected and pixels G01, G10, G34, and G43 at the lower right and the upper left. When the difference calculation in this direction is abbreviated as Sla:G′(Sla)=(G12+G21+G23+G32−G01−G10−G34−G43)/2
FIG. 57(b) shows a difference calculation using the pixels surrounding the pixel R22 to be corrected and pixels G03, G14, G30, and G41 at the upper right and lower left. When the difference calculation in this direction is abbreviated as Bac:G′(Bac)=(G12+G21+G23+G32−G03−G14−G30−G41)/2
FIG. 57(c) shows a difference calculation using pixels surrounding the pixel R22 to be corrected and pixels G10, G30, G14, and G34 in the horizontal direction. When the difference calculation in this direction is abbreviated as Hor:G′(Hor)=(G12+G21+G23+G32−G10−G30−G14−G34)/2
The vertical line flaw detection pattern unit 22 supplies these three difference calculation values to the calculation pattern selector 24.
FIG. 58 shows a calculation in the interpolation pattern unit 20. FIG. 58(a) is a calculation using pixels R20 and R24 in the horizontal direction of the pixel R22 to be corrected and:R22−(R20+R24+G′(Hor))/2
FIG. 58(b) is a correction calculation using the pixels R00 and R44 at the upper left and the lower right of the pixel R22 to be corrected and:R22=(R00+R44+G′(Sla))/2
FIG. 58(c) is a correction calculation using pixels R04 and R40 at the upper right and lower left of the pixel R22 to be corrected and:R22=(R04+R40+G′(Bac))/2
The interpolation pattern unit 20 supplies these three correction values to the calculation pattern selector 24. The interpolation pattern selector 24 compares the magnitude of the three difference calculation values from the vertical line flaw detection pattern unit 22. Then, the interpolation pattern selector 24 determines:ABS(G′(Hor))<ABS(G′(Sla)) and ABS(G′(Hor))<ABS(G′(Bac))   (4)
When the difference value in the horizontal direction is small and the condition (4) is satisfied, the interpolation pattern selector 24 selects the correction pattern of FIG. 58(a). When, on the other hand, the condition (1) is not satisfied, the interpolation pattern selector 24 determines:ABS(G′(Sla))<ABS(G′(Bac))   (5)
When the difference in the horizontal direction is large but the difference in Sla is small, and thus the condition (5) is satisfied, the interpolation pattern selector 24 selects the correction pattern of FIG. 58(b). When, on the other hand, the condition (5) is not satisfied, the correction pattern of FIG. 58(c) is selected.
When the point flaw and the vertical flaw are sequentially corrected as described above, correction circuits for both processes are required, and, moreover, the correction process requires a long time before all defects are corrected. Therefore, it is desirable that the point flaw and the vertical flaw be simultaneously processed. When, however, the point flaw and the vertical flaw are simultaneously processed in the related art as described above, there is a problem in that the precision of the correction is degraded when the point flaw and the vertical flaw are present adjacent to each other.
For example, in a case in which a point flaw is present, in addition to the vertical flaw shown in FIG. 52, in pixels G12 and G14 which are adjacent pixels of the vertical flaw and the pixel values in the pixels are as shown in FIG. 59, that is, the pixel values in the pixels G12 and G14 which are point flaws are 255 in 256 gradation system from 0 to 255, the pixel value in the pixel G21 is 158, the pixel value in the pixel G25 is 217, the pixel value in the pixel G32 is 183, and the pixel value in the pixel G34 is 212. In this case, the four difference calculation values shown in FIG. 54 are:G(Sla)=ABS(G14−G32)=72;G(Bac)=ABS(G12−G34)=43;G(Hor)=ABS(G21−G25)=59; andG(Ver)=ABS(G12+G14−G32−G34)=115
Because G(Bac)<G(Hor), the calculation pattern selector 16 selects the correction pattern of FIG. 55(d) and the pixel G23 is corrected using pixels G12 and G34. However, because a point flaw is present in the pixel G12, the pixel G23 can be corrected only when the pixel G12 has an accurate pixel value.
The above-described problem may occur not only when the point flaw and the vertical flaw are simultaneously corrected, but also when the G pixel, R pixel, and B pixel are simultaneously corrected and when the vertical flaw and a horizontal flaw are simultaneously corrected in a case in which the horizontal flaw and vertical flaw may simultaneously occur because of a gate switch defect of a CMOS.